Repository navigation
Report native Weaver test automation exclusion declarations - #12948
calvinbayer wants to merge 4 commits into
Conversation
2912b77 to
2049459
Compare
8e6e694 to
1153c86
Compare
2049459 to
ba9a78e
Compare
1153c86 to
8091184
Compare
CI Visibility Test Environment - sbt-scalatestJob Status: 🟢 success
Baseline: median of Tests run to verify that CI Visibility behavior has not regressed in the current PR. |
CI Visibility Test Environment - nebula-release-pluginJob Status: 🟢 success
Baseline: median of Tests run to verify that CI Visibility behavior has not regressed in the current PR. |
CI Visibility Test Environment - netflix-zuulJob Status: 🟢 success
Baseline: median of Tests run to verify that CI Visibility behavior has not regressed in the current PR. |
CI Visibility Test Environment - pass4sJob Status: 🟢 success
Baseline: median of Tests run to verify that CI Visibility behavior has not regressed in the current PR. |
CI Visibility Test Environment - reactive-streams-jvmJob Status: 🟢 success
Baseline: median of Tests run to verify that CI Visibility behavior has not regressed in the current PR. |
CI Visibility Test Environment - heliboardJob Status: 🟢 success
Baseline: median of Tests run to verify that CI Visibility behavior has not regressed in the current PR. |
CI Visibility Test Environment - sonar-kotlinJob Status: 🟢 success
Baseline: median of Tests run to verify that CI Visibility behavior has not regressed in the current PR. |
CI Visibility Test Environment - jolokiaJob Status: 🟢 success
Baseline: median of Tests run to verify that CI Visibility behavior has not regressed in the current PR. |
CI Visibility Test Environment - okhttpJob Status: 🔴 failed
Baseline: median of Tests run to verify that CI Visibility behavior has not regressed in the current PR. |
CI Visibility Test Environment - spring_bootJob Status: 🔴 failed
Baseline: median of Tests run to verify that CI Visibility behavior has not regressed in the current PR. |
🟢 Java Benchmark SLOs — All performance SLOs passed
PR vs. master results
Commit: Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion. |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 809118440e
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
CI Visibility Test Environment - sonar-javaJob Status: 🔴 failed
Baseline: median of Tests run to verify that CI Visibility behavior has not regressed in the current PR. |
There was a problem hiding this comment.
Weaver suites with customized names lose their declared exclusion metadata because snapshot capture and reporting use different suite identities. The mismatch also prevents snapshot cleanup at suite completion.
🤖 Bits Code Review · Commit 8091184
8091184 to
df5a259
Compare
6fb7986 to
968588d
Compare
df5a259 to
8cce0bd
Compare
There was a problem hiding this comment.
The shared object-factory retry test fails on TestNG 6.4 and 7.0.0 because it loads an unavailable interface before checking compatibility. Both configurations need to skip this unsupported case.
🤖 Bits Code Review · Commit df5a259
968588d to
fcc0cd5
Compare
8cce0bd to
0ba545b
Compare
|
@autotest Review |
There was a problem hiding this comment.
More details
Exclusion metadata uses Weaver’s task identity, including provider-backed suites, while preserving native failure and skip outcomes. The static assessment supports reporting these declarations without enabling unsupported automation capabilities.
🤖 Bits Code Review · Commit 0ba545b
fcc0cd5 to
6784b3b
Compare
0ba545b to
1c90131
Compare
|
@autotest Review |
There was a problem hiding this comment.
The new callback hook may omit exclusion metadata for Scala 2 Weaver cross-builds because it assumes an instance-method layout. The module’s Scala 3-only coverage does not resolve that compatibility concern.
🤖 Bits Code Review · Commit 1c90131
|
@autotest Review |
There was a problem hiding this comment.
More details
Declaration-tag capture preserves Weaver’s native outcome handling. Source and bytecode inspection supported callback compatibility across the checked Weaver and Scala versions, with no actionable defects identified.
🤖 Bits Code Review · Commit eefd933
6784b3b to
55663bc
Compare
eefd933 to
948fa24
Compare
|
@autotest Review |
There was a problem hiding this comment.
More details
Source and dependency-bytecode inspection support exclusion-tag capture, lookup, and cleanup across the inspected Scala variants, including Weaver 0.8.4 and 0.13.0.
🤖 Bits Code Review · Commit 948fa24
What Does This Do
Report
test.test_management.no_auto_retryandtest.test_management.prevent_failure_suppressionfor native WeaverTestName.taggeddeclarations using the publicTestAutomationExclusionsconstants. Capture a fresh string-only plan snapshot for each suite execution and clean it up when the suite finishes or reporter stops.Motivation
Keep exclusion declaration metadata visible for Weaver, including reported native failure and skip outcomes. Weaver currently supports none of Datadog ATR, EFD, Attempt to Fix, quarantine, or disabling; this change does not enable unsupported automation capabilities.
Additional Notes
Stacked on
codex/munit-automation-exclusions. Validation: module spotlessApply and instrumentedWeaverTest.test automation exclusions*passed, proving both true-only annotation tags on failure and developer-skip spans with unchanged native status. No refactor commit was needed.Snapshot dependency for inherited physical source paths: DataDog/test-environment#604.
test-environment-branch: codex/java-inherited-test-source-snapshots